Breast cancer detection and diagnosis benefit significantly from multi-modal imaging. Mammography (MG) is the first line modality for population screening. For surveillance of women which are known to have an increased risk of developing breast cancer, e.g. due to a family history of cancer or genetic predisposition, additional modalities such as contrast-enhanced magnetic resonance imaging (MRI) are utilized and have been widely integrated into regular healthcare provision programs. Both modalities provide complementary diagnostic information. Several signs of breast cancer like micro calcifications can be identified in MG images but barely in MRI. Other tumors can be detected in mammograms, but the full spatial extent of the tissue disorder is only apparent in the contrast-enhanced MRI images. On the other hand some cancers show-up in MRI, but expose none or only subtle manifestations in MG. Also assessment of the likelihood of malignancy of tumors benefits from multi-modal imaging approaches. For instance, co-localization of micro calcification in MG with tissue angiogenesis as depicted by MRI substantiates positive assessment of tumors. In summary, it is essential to combine the complementary information for detection and assessment of findings which is the basis for clinical decision making and eventual treatment.
The visual appearance of a finding in different modalities is usually highly variable and to a certain extent unpredictable due to the individual tissue physiology, but also because of the different technical principals of each imaging technique. In mammography two-dimensional projection images are acquired from e.g. a cranio-caudal (CC) and mediolateral-oblique (MLO) perspective with the patient in a standing position and the breast significantly compressed to emphasize certain tissue patterns. The image intensity mainly correlates with the radio-opacity of the tissue. In MRI the patient is placed in a prone position on the scanner table, the breast is gently compressed for the purpose of reducing motion artifacts and a contrast agent is administered to improve the contrast in the acquired three-dimensional images. Image intensities in contrast-enhanced MRI correlate in the first instance with the local concentration of contrast agent and protons.
Due to the different nature of the images acquired with different modalities and the physiological variability of tissue, the spatial correlation of different modalities becomes a difficult and time-consuming task even for well trained experts. If a finding has been identified in one modality, but is not immediately apparent in a second modality, the reader has to orient himself by means of anatomical landmarks which are visible in both modalities. The nipple, pectoral muscle, skin surface, major blood vessels or components of the glandular disc may serve as landmarks and help radiologist in finding an approximate corresponding position in a second modality where he/she can perform a local search for additional subtle signs of a tissue disorders. In either situation, the reader has to develop a mental model that takes into account the underlying technical principals of each modality in order to translate positions of findings from one modality to the other modality. Furthermore, a certain level of interaction with the image data is required, e.g. for selecting a correct image slice or changing the orientations of a reformation plane if the modality provides a three-dimensional image volume which is displayed as a two-dimensional image.
Computer tools which support the reader in finding corresponding regions in mammography and MRI images of the same breast promise to improve this workflow in terms of speed and confidence. The article “An intensity-based approach to X-ray mammography—MRI registration” by Mertzanidou et al., Proc. SPIE Medical Imaging: Image Processing, 7623-106, 2010, describes an approach for translating positions in MRI to approximate positions in a CC mammogram of the sane breast. Central component is a finite element model (FEM) of the biomechanics of the breast. Using a FEM that is adapted to the individual patient using a segmentation of the breast in the MRI image, the same compression force applied in mammography is applied in a simulation to the MRI image. Subsequently, MRI intensities are mapped to X-ray attenuation values and a two-dimensional projection image is simulated from the deformed MRI dataset. The simulated X-ray mammogram resembles the real mammogram and can therefore be aligned with the latter using conventional intensity-based image registration algorithms. This last step completes the processing pipeline which allows for translating three-dimensional positions in MRI to the corresponding two-dimensional position in CC mammograms. In the opposite direction, a two-dimensional position in the CC mammogram can only be translated into a line in the MRI due to the missing depth information in the X-ray projection image. This rather sophisticated approach has the disadvantage that FEM are computational very demanding and require reasonable approximations of the biomechanical properties of breast tissue. Additionally, the method requires a detailed segmentation not only of the pectoral muscle and skin surface but also of the fatty and glandular tissue within the breast for the purpose of estimating reasonable X-ray attenuation values from MRI intensities.
The articles “Fusion of contrast-enhanced breast MR and mammographic imaging data” by C. P Behrenbruch et al., The British Journal of Radiology, 77 (2004), S201-S208; “MRI—Mammography 2D/3D Data Fusion for Breast Pathology Assessment” by C. P Behrenbruch et al., Proceedings of MICCAI 2010 and “Fusion of contrast-enhanced breast MR and mammographic imaging Data” by C. P Behrenbruch et al., Medical Image Analysis 7 (2003) 311-340 utilize a combination of pharmacokinetic modeling, projection geometry, wavelet-based landmark detection and thinplate spline non-rigid registration to transform the coordinates of regions of interest (ROIs) from two-dimensional mammograms to the spatial reference frame of contrast-enhanced MRI volumes.
The article “Two-dimensional three-dimensional correspondence in mammography” by R Marti et al., Cybernetics and Systems, Volume 35, Number 1, January-February 2004, pp. 85-105(21) also simulates X-ray projection images from the MRI data that resemble the original MLO and CC mammograms. The MR-based projection images are further deformed using rigid intensity-based registration followed-by a matching of local salient points from linear structures as well as anatomical boundaries of the MR-based projection images with the true original mammograms.